Communication over data networks has become commonplace. Examples of such communications include the use of packet-based digital data to communicate audio and video data in real-time over a data network. Voice and video data sharing, for example, can occur in real time over a data network between two or more users during a streaming videoconference or other virtual meeting.
Required bandwidth to transport the data can be considerable. Taking video conferencing as one example application, multiple participants may be streaming voice, video, and application data with each of the other participants in real time. Each participant may receive all data streams from all other participants. Each participant may be connected to the conference by a network connection that has a different bandwidth capacity from all other connections. Further, other data traffic is often being carried over the network connections that may be unrelated to the videoconference.
To reduce the amount of bandwidth required, compression of data prior to communication may be performed. Software and/or hardware components that perform these steps may be referred to as “coders” and/or “decoders,” and are more commonly known as “codecs.” Codecs may perform other tasks in addition to data compression. Depending on the particular steps that a codec performs, the amount of data to be processed, and other factors, codecs may consume significant processor resources. In the particular case of a video codec, for instance, the demands that a codec places on a processor varies with the size and transmission rate of images. Large images that are communicated at a high frequency require more extensive use of a codec than smaller images communicated at a lower frequency.
Some steps of data compression performed by codecs are known. Depending on the particular type of data, different compression steps may occur of varying computational complexity. Because video data can be quite voluminous, various schemes of compression have been developed unique to video data. Both temporal and spatial compression may be performed. One example compression scheme is discrete cosign transformation (DCT) compression. Another is referred to as motion detection, or motion vector analysis. Generally, motion detection operates to determine which portion of a video image has changed from a previous image (i.e., what has “moved”). Only the portion of the image that has changed is then communicated, as opposed to the entire image. For example, if a person is speaking in front of a camera, the image from frame to frame may include an unchanging background, with only the speaker's face showing differences. In this case only the pixels of the image that make up the face may be compressed and communicated, with the background remaining constant from frame to frame.
Motion detection may be practiced in many different particular steps. Some codecs perform a detailed comparison of frames to determine change (e.g., a codec may “subtract” a second frame from a first to identify differences). Others perform other steps of analysis that may include additional opportunities for data savings. Use of motion vectors, for example, can further reduce the amount of data required to be communicated. In addition to searching an image only for changed portions (i.e., movement), motion vector analysis further attempts to identify a portion of the image that has changed, motion analysis steps may include identifying only the portion of a previous frame that has moved and simply code instructions to duplicate that portion in a new position in a subsequent frame. This may be accomplished, for instance, by calculating motion vectors that describe the movement.
For example, if a speaker is sweeping his arm across the screen as he speaks, a codec performing motion vector analysis may identify that the speaker's arm is shifting from frame to frame and determine the frame by frame shift. Rather than sending updated portions of the image data that include the arm in its new position, the codec will identify the portion of the previous frame that includes the arm and specify motion vectors that describe the direction and distance to shift the arm in the a sequential frame.
Compression steps such as motion vector analysis can be computationally intensive. A search and comparison of frames must be done to identify shifted portions. Further calculations must then be made to derive motion vectors that describe the direction and amount of shift. Generally, motion vector analysis is performed in 1, 2, 4, 8, 16, etc. pixel search areas. The larger the search area, the more computations are required.
These and other steps of video data compression can place extreme demands on processors. Potentially, the processor can reach maximum utilization and cause application failure. Further, even if a maximum level of processor utilization is not reached, excessive video compression calculations can cause a processor to neglect other applications unrelated to the codec that are simultaneously being run. These problems are troublesome for videoconference codecs that may be used by a variety of different machines that utilize different processors and different secondary applications running thereon. In a videoconference with multiple attendees, for example, some participants may be using personal computers with relatively slow processors that must support numerous other simultaneous services and applications, while other participants may have dedicated servers with much more powerful processors. Some of the processors may be able to support intensive video compression analysis, while others of the machines can support only minimal compression.